Heaven Test Stand: Towards Comparative Research on RSP Engines

The benchmarking of window-based RDF Stream Processing (RSP) engines has recently attracted the attention of the Stream Reasoning community. Solutions like LSBench, SRBench and CSRBench tried to fulfill the need of shared practices for RSP engine evaluations. However, an infrastructure for the systematic comparison of existing systems is still missing. In this paper, we propose the requirements and an architecture for Heaven Test Stand, a facility to foster Systematic Comparative Research Approach (SCRA) for window-based RSP engine. Heaven allows to design and systematically execute repeatable, reproducible and comparable experiments. As further contribution a working implementation of Heaven Test Stand is released as open source.